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THE EFFECTS OF PRACTICING WITH A VIRTUAL ULTRASOUND TRAINER 



ON FAST WINDOW IDENTIFICATION, ACQUISITION, AND DIAGNOSIS 

Gregory K. W. K. Chung, Ruth G. Gyllenhammer, and Eva L. Baker 

Abstract 

In this study, we compared the effects of simulator-based virtual ultrasound scanning 
practice to classroom-based hands-on ultrasound scanning practice on participants’ 
knowledge of FAST window quadrants and interpretation, and on participants’ 
performance on live patient FAST exams. Twenty-five novice participants were 
randomly assigned to the simulation-practice condition and 24 participants to the 
classroom-practice condition. Participants were mostly medical school students, with 
some nursing students. Participants who received simulation-based practice scored 
significantly higher on interpreting static images of FAST windows. On live patient 
exams where participants scanned the RUQ, LUQ, and suprapubic quadrant of a normal 
patient and an ascites positive patient (6 quadrants total), there were no statistical 
differences between the two conditions on scan time, window acquisition, and window 
interpretation except for the following: the classroom-practice condition had shorter scan 
time for the LUQ, and a higher number of participants attaining high window quality on 
the RUQ (normal patient only) and suprapubic quadrant (positive patient only) and 
correct window interpretation on the LUQ (normal patient only). Performance was 
statistically similar on all other window quality and window interpretation measures on 
the other quadrants. Overall, classroom-based practice appeared to promote physical 
acquisition skills (e.g., acquiring a window) and simulator-based practice appeared to 
promote window interpretation skills. The simulator used (SonoSimulator™) was well 
received by participants and appears promising as a training tool not only to increase 
probe time, but also to increase exposure to FAST windows reflecting various anatomy 
and disease states. 



INTRODUCTION 

Unfamiliarity with ultrasonography, the cost of training users on ultrasound-guided 
procedures, and the lack of training opportunities are limiting the use of this potentially 
beneficial technology. A cost-effective and widely applicable method for providing users 
with ultrasound-guided procedural training is needed. Purported simulator-based training 
advantages over traditional medical training include: (a) no risk to trainees or patients during 
practice attempts; (b) more cost-effective than current training methods; (c) provides multiple 
modes of sensory interaction to maximize learning; (d) user gains and maintains proficiency 
through unlimited repetition (avoids skill decay); (e) independent, self-directed learning; and 
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(f) provides a method for performance tracking. In this study, we compared simulation-based 
practice of ultrasound scanning to classroom-based practice of ultrasound scanning on 
knowledge and performance measures. We focused on one type of procedure, the focused 
assessment with sonography for trauma (FAST) exam, as the context for the comparison. 

Focused Assessment with Sonography for Trauma (FAST) Exam 1 

The FAST exam is an emergency ultrasound procedure which focuses on the detection 
of free fluid: hemoperitoneum, hemopericardium, pneumothorax, and hemothorax (Fox & 
Irwin, 2008). Unlike other trauma screening modalities such as the physical exam, diagnostic 
peritoneal lavage, and the CT scan, the FAST exam is non-invasive, bedside, and repeatable; 
requires five minutes to complete; and does not require a stable patient, making it invaluable 
as an initial assessment (Brooks, Davies, Smethhurst, & Connolly, 2004; Fox, 2010b; 
Lingawi & Buckley, 2000). 

The standard windows of the FAST exam are the right upper quadrant (RUQ), left 
upper quadrant (LUQ), sub-xiphoid (or subcostal), and suprapubic. The quadrants are 
examined for fluid in the gutters or space between the organs in each window. In the RUQ 
(also called Morrison’s Pouch), evidence of fluid may be found between the liver and the 
kidney. Fluid may also be found between the kidney and spleen in the LUQ view or in the 
pelvis in the suprapubic view (Salen, Melanson, & Heller, 2000). For the purposes of our 
study, the FAST exam was limited to the RUQ (Morrison’s Pouch), LUQ (spleen), and the 
suprapubic (bladder) windows, as the model patients were ascites positive with free fluid in 
only these quadrants. 

Virtual Patient and Ultrasound Scanning Simulator 

The simulator system we used, SonoSimulator™, was developed by Pelagique. The 
virtual patients used in SonoSimulator™ were modeled with real patient ultrasound scans of 
both normal and pathologic cases with a generic ultrasound design. The pathology in this 
study was limited to ascites; thus, fluid in the cardiac window was not available to 
participants. The SonoSimulator™ database contained a range of cases with varying amounts 
of free fluid: absence of fluid, and minimal, moderate, and severe amounts of fluid. The 
variety of patients used to populate the virtual patient database also provided scans with a 
range of anatomy. The system is PC-based, making it both widely accessible and 
transportable at a low cost. 



'Also referred to as focused abdominal sonography for trauma exam. 
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Cognitive Demands of FAST Window Acquisition and Interpretation 



Recognition of anatomical landmarks is fundamental to ultrasound window acquisition. 
Both diagnostic medical sonographers and emergency physicians require training and 
proficiency in abdominal anatomy as a component of medical training and as a professional 
skill. To become credentialed by the American Registry for Diagnostic Medical Sonography 
and certified by the American Registry of Radiologic Technologists, sonographers must pass 
examinations requiring knowledge of normal and abnormal anatomy and physiology. The 
Ultrasound Task Force of the Society for Academic Emergency Medicine (SAEM), in their 
model curriculum for emergency physicians training in ultrasonography, also heavily 
emphasizes anatomy (Mateer et al., 1994). 

Ultrasound scanning involves manipulating a probe or transducer against the patient’s 
body, wherein the probe essentially acquires a 2-dimensional slice through a 3 -dimensional 
anatomical volume. Minute changes to the pitch, yaw, and roll of the probe causes changes in 
the window image, and facile probe manipulation is required to acquire clear anatomical 
landmarks. This skill is not easily taught. Experienced sonographers use nuanced hand 
movements instinctively during image acquisition (Weidenbach et ah, 2005). Precise verbal 
instructions relating the trainer’s actions to the image on screen can be difficult. The FAST 
exam and ultrasonography in general is “operator dependent” (Robinson, 2000). To make a 
correct diagnosis, the sonographer must first acquire an adequate window by using 
anatomical landmarks and probe manipulation techniques such as fanning. Complicating the 
task is that the detection of fluid may depend on positioning of either the patient or the probe, 
with subtle changes in either required for precise window analysis (American Institute of 
Ultrasound in Medicine [AIUM], 2008; Salen et ah, 2000). 

Use of Ultrasound Simulators for Training 

The goals of FAST training are for the trainee to be able to accurately diagnose the 
presence of free fluid, acquire an adequate window to do so, and identify both the free fluid 
and the window when given a single ultrasound scan. Because window acquisition is 
dependent on probe movement, physical probe time is thought to be absolutely necessary to 
training. A typical curriculum for the FAST exam includes both didactic and hands-on 
instruction. Didactic training presents instruction on the principles of ultrasonography, an 
introduction to ultrasound mechanics or knobology, and discussion of the purpose, method, 
and interpretation of the FAST exam. The hands-on training component is the practical 
application of instruction and also the most constrained due to limitations of practicing with 
model patients (Salen et ah, 2000; Shackford et ah, 1999). 
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Ideally, a trainee would gain experience by performing the FAST exam on as many 
individuals, or cases, as possible. However, extended training experience with model patients 
is rarely practical. For example, to scan the LUQ, RUQ, and suprapubic quadrants would 
take 5 to 8 minutes in a training context per trainee. Assuming a typical class size of 20 
students, with each student given two opportunities to scan, the total time required would be 
between 3 and 5 hours. Thus, providing trainees with the opportunity to practice scanning on 
different patients (to vary the anatomy) with different free fluid states (to vary the severity), 
and to scan multiple times to attain proficiency remains largely impractical. 

One of the greatest benefits of simulators may the capability for the simulator to 
provide interactive learning opportunities with a large number of cases. Previous research 
points to the effectiveness of the simulator training in preparing trainees to both perform and 
interpret the FAST exam. For example, in a four-hour FAST training course for emergency 
medicine resident physicians, the UltraSim sonographic (mannequin) training model was 
found to be as effective in preparing trainees to detect the presence of intraperitoneal free 
fluid in various FAST windows as the training with a live patient model (using peritoneal 
dialysis patients) (Salen et al., 2000). In another study, the use of the SonoTrainer 
sonography simulation system enabled physicians trained with the system (versus theoretical 
training alone) to make diagnoses of second-trimester fetal abnormalities with a detection 
rate of 86% and specificity of 100% (Maul et al., 2004). 

Thus, one advantage of simulation-based training systems for ultrasound is the 
capability to maximize probe time, thus aiding the acquisition of nuanced movements and 
virtual imaging essential in both window acquisition and identification. Various systems have 
been developed with a range of capabilities (e.g., UltraSim, VirUS, EchoComJ, and 
SONOSim3D, SonoTrainer, Maul et al., 2004). For the FAST exam, the simulator used in the 
current study (SonoSimulator™) focuses on providing trainees the capability to visualize free 
fluid in two ways: as a single ending image (e.g., snapshot of the window) and as a 
continuously changing window reflecting the process of window acquisition. The simulator 
can provide real-time imaging and visual feedback of the anatomy and gutters across 
multiple cases, allowing the trainee to be exposed to far more cases than would be possible in 
typical classroom-based training with live patients. However, there is little prior research that 
compares simultaneously simulator-based practice to classroom-based practice with respect 
to knowledge of given FAST windows and on execution of FAST exam procedures with live 
patients. 
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Research Question 



This study focused on addressing the extent to which practice using an ultrasound 
simulator and virtual patient affected participants’ knowledge and subsequent performance 
on live FAST exams, compared to practice in a typical classroom context with a real 
ultrasound machine and live model patient. The training manipulated was the practice 
component of FAST scanning. The main outcomes were (a) knowledge of window 
identification and window interpretation, and (b) performance of a FAST exam with normal 
and ascites positive patients. 



METHOD 

Design 

A pretest/posttest control group design was used to examine the effects of the two 
practice conditions on participants’ knowledge of and performance on a FAST exam. 
Participants in the control condition received guidance and feedback from an instructor and 
hands-on practice with a model patient. Participants in the experimental condition practiced 
scanning on virtual patients using an ultrasound simulator, and received quality of scan and 
accuracy of diagnosis feedback from an ultrasound expert. 

Sample 

Forty-nine participants participated in this study. The mean age was 25.61 years 
(SD = 3.19 years). Twenty-four participants were assigned to the group practice condition 
and 25 to the simulation practice condition. Forty-four participants were medical students, 
three were nursing students, one was a medical resident, and one was an undergraduate. The 
mean MCAT score was 33.63 ( SD = 3.22). Nineteen participants reported no prior training in 
ultrasound scanning procedures, 15 reported a lecture or classroom training, and nine 
reported some hands-on practice. Overall, the sample represented novices with limited or no 
experience with ultrasound scanning. 

Model Patient Selection 

To (visually) replicate the abdominal condition of patients with blunt trauma resulting 
in hemorrhage, patients with ascites were recruited to serve as model patients. Ascites is a 
pathological condition resulting from liver disease wherein fluid accumulates in the 
abdominal cavity (Runyon, 1994; Yu & Hu, 2001). For the first two waves of data collection, 
the same model patient (moderate degree of free fluid) was used. Scheduling conflicts 
required the use of a second model patient for the third data collection wave. This model 
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patient had severe degree of free fluid. Both patients were positive for free fluid in the right 
upper quadrant, left upper quadrant, and suprapubic regions. 

Tasks 

There were four major tasks participants engaged in: (a) answering paper-based 
questions related to knowledge of the FAST exam; (b) viewing computer-based training 
videos on the physics of ultrasound and the FAST exam procedures; (c) receiving hands-on 
practice with a model patient in a group-based setting typical of ultrasound training, or 
receiving practice scanning virtual patients with an ultrasound simulator; and (d) conducting 
a FAST exam on two model patients (one normal and one with ascites). 

Knowledge Measures 

The knowledge measures were embedded in a 74-item pretest, which included items 
related to the FAST exam and participants’ knowledge of abdominal anatomy. The posttest 
contained the same FAST exam questions (but not the anatomy questions), and in addition, 
questions related to participants’ background. Appendix A contains a copy of the posttest, 
which includes the background questions, and Appendix B a copy of the pretest anatomy 
items. In addition, we also asked participants for their feedback on the practice tasks 
(Appendix C and Appendix D). 

Instructional Videos 

Participants viewed two instructional video modules in sequence (Fox, 2010a, 2010b). 
The first video was on the physics of ultrasound and general principles of sonography. 
Topics covered in this module include: ultrasound instrumentation and image acquisition — 
mode, Doppler, echogeneity, transducer basics, image orientation, system controls (depth and 
gain), and artifacts (high attenuation, low attenuation, gas scatter, refraction, reverberation, 
mirror). The second video focused specifically on the FAST exam, describing how to view 
and interpret each of the four window quadrants: right upper quadrant (RUQ), left upper 
quadrant (LUQ), suprapubic (pelvis), and sub-xiphoid (cardiac). The video demonstrated a 3- 
minute FAST exam, discussed the implications of the mirror image artifact, discussed major 
pitfalls, and provided case study examples of conditions wherein free fluid is present. 

FAST Exam Practice 

Classroom-based practice. In the control condition, the instructor gave participants 
five minutes of instruction on machine operation basics followed by a demonstration of the 
FAST exam. Each participant then practiced the FAST exam on the model patient while the 
instructor provided guidance and feedback (RUQ, LUQ, suprapubic quadrant). A normal 
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model patient was used to demonstrate anatomic landmarks, consistent with the established 
training procedure used by the instructor and by other FAST educators and practitioners 
(Salen et al., 2000; Salen et al., 2001). Each participant was given two opportunities to 
conduct the FAST exam. One hour was allotted for the classroom practice session. Appendix 
E contains a picture of a classroom practice session, Appendix F contains a picture of the lab 
setup, and Appendix G and Appendix H contain copies of the observation forms used during 
classroom practice and during the performance test. 

Simulator-based practice. In the experimental condition, participants were given a 
brief introduction to the simulator (number of cases; software features — freeze, layers, pop- 
up windows; and indicator/transducer positioning). For the second and third data collection 
waves, prior to the start of the simulation task, the sonographer gave brief instructions on 
locating landmarks on the body. The simulator was laptop-based and presented up to 10 
cases with varying levels of normality and severity. Participants manipulated a probe, and the 
probe movements were mapped to corresponding ultrasound scans. The scans were updated 
in real-time and appeared as if the participant were using a real ultrasound machine. 
Appendix I contains a picture of a simulation practice session, Appendix J contains a copy of 
the simulator user interface, and Appendix K contains a copy of the simulation task 
directions. 

Each case in the simulation was subdivided into three views: LUQ (Morrison’s Pouch), 
LUQ (spleen), and suprapubic (bladder). Participants were told to find the ideal diagnostic 
window and then freeze the scan. Participants were then instructed to indicate their diagnosis, 
window quality confidence level, and diagnosis confidence level on a pop-up window. A 
report with the participant’s window, diagnosis, confidence levels, time to scan, and start and 
end times was then printed out and given to an expert sonographer for evaluation. The expert 
sonographer evaluated the students’ diagnosis and window quality and determined whether 
the student should advance to the next view. Criteria for advancement were an accurate 
diagnosis and an excellent or fair window. No other instruction was given by the 
sonographer. One hour was allotted for the simulator session. Appendix L contains a copy of 
the ultrasound simulation report. 

FAST Performance Test 

After practice, participants were given a performance test. The performance test 
required participants to conduct a FAST exam on two model patients. During the test, no 
help or feedback was given to the participants. 
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Participants examined one normal patient and one patient with ascites pathology using 
a portable ultrasound machine. The models used were the M-Turbo (SonoSite, 2010a), 
MicroMaxx (SonoSite, 2010b), and S-FAST (SonoSite, 2010c). A researcher readied the 
transducer and began timing when the transducer made first contact with the patient. Timing 
stopped when the participant indicated completion of the scan or when the participant 
indicated an inability to find an adequate window. After each window capture, the 
participants were asked to render a diagnosis. Images captured were saved to disk and later 
evaluated by an expert sonographer. Participants were limited to 2 minutes each for the RUQ 
and suprapubic scans, and 4 minutes for the LUQ. 2 Participants scanned patients in order of 
patient availability to expedite the testing process, minimize patient discomfort, and 
counterbalance model patient exposure to participants. Appendix M contains a copy of the 
sonographer evaluation form. 

Measures 

Two major types of measures were developed for this study: (a) knowledge-based 
measures used to evaluate participants’ knowledge of FAST-exam-related concepts; and 
(b) performance-based measures used to evaluate participants’ skill at executing a FAST 
exam. 

Development of Knowledge-Based Measures 

Knowledge measures were derived from information covered in the instructional videos 
(Fox, 2010a, 2010b) as well as FAST-exam- specific concepts. Three broad areas were 
sampled: prior knowledge of anatomy, basic FAST exam procedures, and window 
interpretation. Knowledge of the abdominal area was targeted because interpretation of 
FAST scans requires proficiency with abdominal anatomy. Knowledge of the FAST exam 
procedures was used to measure both the existing knowledge of participants prior to 
instruction on FAST concepts, and how much knowledge participants acquired from the 
instruction. Finally, window interpretation was sampled the most because this represented the 
most important knowledge outcome of the training. 

The following types of sources were consulted to develop or adapt items for the 
knowledge measures: instructional materials (e.g., textbooks, guidebooks, and research 
[Alberto, Kelleher, & Nutt, 2007; Carnes, 2007; Noble, Nelson, & Sutingco, 2007; Reardon, 
2008; Tempkin, 2009]), and computer-based instruction used in the study (Fox, 2010b). Draft 



2 These time limits were established after the first data collection wave, as one participant was taking a very 
long time to complete an LUQ scan. The thresholds were based on the observed timing of participants during 
the first data collection wave and consultation with sonographers. 
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items were reviewed for accuracy and interpretability by a director of emergency ultrasound, 
director of ultrasound and breast imaging, and a director of an emergency ultrasound 
department. The review resulted in revisions to 11 items. In addition, four experts 
experienced in sonography (director of ultrasound and breast imaging, experienced 
sonographer, and two emergency physicians) were administered the pretest measure. Based 
on their responses, two items were dropped. Table 1 shows the final distribution of the 
pretest items by topic. 



Table 1 



Knowledge-Based Measures 



Knowledge 


No. of 
items 


Pretest form question 
number 


Prior knowledge of anatomy 


Identification of abdominal organs 


8 


5, 13a- 13g 


Basic FAST scanning procedures 


Scanning planes 


6 


6, 7, 8, 15a -15c 


Ultrasound probe placement 


5 


11, 12, 16a- 16c 


Echogeneity 


2 


3,4 


Artifact recognition 


2 


33,34 


Free fluid 


2 


9, 10 


Window interpretation 


Anatomical interpretation of FAST windows 


16 


14a -14j, 17a -17f 


Identification of FAST window quadrants 


14 


18-22, 24 -32 (part a) 


Diagnostic interpretation of FAST windows 


14 


18-22, 24 -32 (partb) 



Pretest of knowledge of anatomy. As a check on prior knowledge, participants were 
asked to identify the organs on an abdominal diagram. This measure was intended as a basic 
check on participants’ knowledge of anatomy of the abdominal region. There were eight 
items in this measure (a = .46, N = 49). 

Knowledge of basic FAST concepts. This measure contained items sampling the 
following concepts: basic definitions related to the FAST window, identification of 
anatomical parts with respect to the FAST window, identification of windows related to 
probe position, and identification of windows and anatomical parts related to the FAST 
window. There were 17 items in this measure (a = .75, n = 45). 
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Anatomical interpretation of FAST windows. This scale measured participants’ 
knowledge of the anatomical component of a given FAST window. There were 16 items in 
this measure (a = .80, N = 49). 

Identification of FAST window quadrants. This scale measured participants’ 
knowledge of the quadrant (LUQ, RUQ, or suprapubic) of a given FAST window. There 
were 14 items in this measure (a = .91, N = 49). 

Diagnostic interpretation of FAST windows. This scale measured participants’ 
knowledge of the disease state (normal, abnormal, inadequate to interpret) shown in a given 
FAST window. There were 14 items in this measure (a = .84, N = 49). 

Development of Performance-Based Measures 

Measures of participants’ performance were designed to gather basic information on 
how long participants took to acquire a window, the quality of the window, and the quality of 
the interpretation given the window. We recorded scan time during practice and during the 
live test. The evaluation of scan quality and diagnosis were performed by experts trained in 
sonography (either emergency medicine physicians or experienced sonographers). 

Performance Measures of FAST Exam: Practice Trials 

During the practice trials participants received different types of practice depending on 
the condition. In the control condition, participants received one-on-one guidance and 
feedback from the instructor, with feedback and elaboration given to the whole group as 
necessary. In the experimental condition, participants received minimal feedback (knowledge 
of results of their scans and diagnoses), which would be typical in a fully automated system. 

Window acquisition time. Window acquisition time was defined as the time required 
for a participant to acquire a window. For the control condition, the acquisition time was the 
period between first contact with the model patient’s body and when the participant said 
“freeze.” For the experimental condition, acquisition time was defined as the start of the 
procedure until the participant pressed the “freeze” button. In both conditions, the 
participant’s judgment of an adequate window or the participant’s judgment that he or she 
could not acquire the window determined the stop time. 

The following measures were available only for the experimental condition. 

Window quality. For each acquired window, an expert evaluated the quality of the 
window. In the experimental condition, an expert reviewed the window and rated the window 
as excellent, fair, or poor, then returned the rating to the participant. 
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Diagnostic interpretation. For each acquired window, the participant rendered a 
diagnosis of that window. An expert sonographer evaluated the diagnosis as correct or 
incorrect and returned the rating to the participant. 

Window and diagnosis confidence levels. Immediately after participants acquired the 
window they were prompted to rate their confidence in their window acquisition and their 
diagnosis of the disease condition. For each factor, participants were asked to indicate, on a 
5-point Likert scale (1 = not confident, 2 = slightly confident, 3 = moderately confident, 4 = 
very confident, 5 = extremely confident ), their confidence in the quality of their window and 
the accuracy of their diagnosis. 

Number of trials to successfully diagnose a case. The number of trials (attempts) 
each participant took to successfully complete a case was recorded. 

Performance Measures of FAST Exam: Live Patient Trials 

During the live trials all participants were measured using the same method. 

Window acquisition time. Window acquisition time was defined as the time in 
seconds required for a participant to acquire a window. The acquisition time was the period 
between first contact with the model patient’s body and when the participant said “stop” to 
indicate an adequate window or the participant’s judgment that he or she could not acquire 
the window. 

Window quality. For each acquired window, an expert evaluated the quality of the 
window. In the first wave of data collection, an expert evaluated the window quality 
immediately after the participant acquired the window. On subsequent data collection waves, 
an expert evaluated the window quality offline. The window was rated as excellent, fair, 
poor, or other. Other captured situations where the window acquired was non-diagnostic. No 
feedback was provided to participants. 

Diagnosis accuracy. For each acquired window, the participant rendered a diagnosis of 
that window. An expert evaluated the quality of the diagnosis. In the first wave of data 
collection, an expert evaluated the diagnosis immediately after the participant rendered the 
diagnosis. On subsequent data collection waves, an expert evaluated the diagnosis after data 
collection. The diagnosis was rated as correct, incorrect, or other. Other captured situations 
where the window acquired was non-diagnostic. Note that correct occasionally indicated 
situations where an excellent scan was acquired and diagnosis based on that scan was correct, 
but due to patient positioning or other variability, a positive window was not acquired. No 
feedback was provided to participants. 
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Supplementary Measures 

Additional measures were administered to gather information on individual differences, 
perceived benefits of the practice sessions and simulator, and general background. 

Usability. Participants were asked several questions about their perception of how 
useful the practice was with respect to probe manipulation, window acquisition, and window 
interpretation. Participants were asked, “How well did the ultrasound practice session (either 
with the simulator or in the group session) prepare you to perform a ‘real’ ultrasound 
examination of a live patient?...” 

1. ... only consider the physical aspect of scanning (i.e., manipulating the probe) 

2. ... only consider the acquisition of a scan window aspect of scanning (i.e., being 
able to acquire a high quality window) 

3. ... only consider the diagnostic aspect of scanning (i.e., being able to identify 
normal or abnormal conditions) 

For each item, participants were instructed to indicate, on a 4-point Likert scale 
(1 = very inadequate amount of practice, 2 = inadequate amount of practice, 3 = adequate 
amount of practice, and 4 = too much practice), their perception of how adequate the practice 
was. 

In addition, participants in the experimental condition were asked questions about their 
perception of how effective the simulation was in preparing them to acquire and interpret 
windows. Participants were instructed to indicate for each item, on a 4-point Likert scale 
(1 = strongly disagree, 2 = disagree somewhat, 3 = agree somewhat, and 4 = strongly agree), 
how much they agreed with each of the following statements: 

1. Practicing with the simulator prepared me to interpret the scans during the live 
patient test 

2. Practicing with the simulator prepared me to acquire the windows during the live 
patient test 

3. The simulator was effective in helping me interpret scans of different kinds of 
conditions 

4. The simulator was effective in helping me interpret scans of the different quadrants 

Participants were also asked (a) What were the most useful features of the simulator? 
(b) How could the simulator be improved? and (c) What would have helped you improve 
your performance conducting the FAST scan on today’s patients? Finally, participants were 
asked about how confident they were with respect to conducting a FAST scan on actual 
patients. 
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Background information. Demographic information and prior experience with 
ultrasound training were gathered. Participants were asked their age, gender, current position, 
and MCAT scores. Participants were also asked what type of prior ultrasound training they 
received and the number of hours spent training on ultrasound procedures. 

Procedure 

Participants were randomly assigned to the experimental or control conditions during 
scheduling. Each condition arrived in waves at predetermined times. One researcher 
introduced the research team and provided an overview of the study (3 minutes). Then the 
pretest of FAST knowledge was administered. Participants generally finished the pretest 
within 20 minutes and all participants were allowed as much time as needed to finish. 
Participants were then given two hours to view instructional videos on the physics of 
sonography and the FAST exam procedure. 

Following the video instruction, participants received practice (either group-based 
practice with one instructor, or simulation practice with the ultrasound simulator). Following 
the practice session, participants were then required to conduct a live FAST exam with two 
patients; acquire the RUQ, FUQ, and suprapubic windows; and render a diagnosis. After the 
live patient exam, participants were given the knowledge posttest, and filled out the feedback 
form and paperwork to receive payment. The entire protocol took around 5 hours. 



RESULTS 

Preliminary Analyses 

Checks of Assumptions 

Measures used in the analyses were checked for normality and outliers, and equality of 
variances was checked for ANOVA-based tests. In general, the measures were normally 
distributed and variances were similar across conditions. Checks were conducted on the 
samples to examine whether there were pre-existing differences between the control and 
experimental conditions, t tests were conducted on self-reported number of hours of prior 
ultrasound training, MCAT scores, prior knowledge of anatomy, and the pretest scales for 
anatomical interpretation, window identification, and diagnostic interpretation of FAST 
windows. These results suggest the conditions were equivalent on their knowledge of 
ultrasound scanning procedures. 
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Main Analyses 



The main research question in this study was to what extent does FAST exam practice 
using a virtual simulator affect participants’ knowledge and subsequent performance on live 
FAST exams, compared to practice in a typical classroom context with a real ultrasound 
machine and live model patient? The training manipulated was the practice component of 
FAST scanning. The main outcomes were (a) knowledge of window identification and 
window interpretation, and (b) performance of a FAST exam with normal and ascites 
positive patients. 



Table 2 

Descriptive Statistics of Knowledge Measures (N = 49) 



Control Experimental 









(n = 


24) 






(n = 


25) 




Measure 


Max. 

possible 


M 


SD 


Min. 


Max. 


M 


SD 


Min. 


Max. 


Identification of abdominal 
anatomy 

Basic FAST scanning procedures 


8 


6.50 


1.22 


3.00 


8.00 


6.57 


1.12 


5.00 


8.00 


Pretest 


17 


6.79 


2.90 


0.00 


11.00 


6.14 


3.21 


0.00 


13.00 


Posttest 


17 


14.25 


1.51 


11.00 


17.00 


14.36 


1.85 


9.00 


17.00 


Anatomical interpretation of 
FAST window 




















Pretest 


16 


5.83 


2.91 


0.00 


13.00 


4.86 


2.35 


0.00 


8.00 


Posttest 


16 


12.83 


1.79 


10.00 


16.00 


12.08 


2.29 


6.00 


16.00 


Identification of FAST window 
quadrant 




















Pretest 


14 


1.75 


2.92 


0.00 


9.00 


1.29 


2.85 


0.00 


11.00 


Posttest 


14 


8.50 


2.40 


3.00 


12.00 


8.92 


2.41 


1.00 


13.00 


Diagnostic interpretation of FAST 
window 




















Pretest 


14 


1.92 


2.89 


0.00 


11.00 


0.86 


1.59 


0.00 


5.00 


Posttest 


14 


8.46 


1.41 


6.00 


11.00 


9.56 


2.29 


2.00 


12.00 
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Table 3 

Descriptive Statistics of Performance Measures (N = 49) 



Control Experimental 







C n = 


24) 






C n = 


25) 




Measure 


M 


SD 


Min. 


Max. 


M 


SD 


Min. 


Max. 


Diagnostic interpretation of FAST 
window a 


4.13 


1.19 


2.00 


6.00 


3.80 


1.12 


2.00 


6.00 


Acquisition of FAST window a 


3.33 


1.58 


0.00 


6.00 


1.92 


1.58 


0.00 


6.00 


Total scan time (sec) 


340.25 


303.66 


84.00 


1644.00 


496.96 


214.47 


180.00 


934.00 


RUQ scan time (sec) 


125.71 


133.04 


17.00 


548.00 


139.76 


62.64 


33.00 


257.00 


FUQ scan time (sec) 


155.00 


132.17 


25.00 


548.00 


269.16 


178.28 


53.00 


673.00 


Suprapubic scan time (sec) 


68.87 


105.76 


22.00 


548.00 


88.04 


47.82 


9.00 


180.00 



a Summed across patients and quadrants. Maximum score is 6. 
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Table 4 



Intercorrelations (Pearson) Among Posttest Measures 



Knowledge measures 



Posttest measure 


1 


2 


3 


Knowledge measures 

1. Basic FAST scanning procedures 

2. Anatomical interpretation of FAST window 


42 ** 






3. Identification of FAST window quadrant 


.34* 


.31* 


- 


4. Diagnostic interpretation of FAST window 


.29* 


.22 


.65* : 


Performance measures 

5. No. of correct FAST window interpretations a 


.17 


.14 


.24 


6 . No. of excellent FAST windows a 


.06 


.15 


.01 


7. Total scan time (sec) 


.15 


.07 


-.07 


8 . RUQ scan time (sec) 


.19 


.01 


.03 


9. LUQ scan time (sec) 


.14 


.08 


-.09 


10. Suprapubic scan time (sec) 


.12 


.12 


-.01 



a Maximum possible is 6 . 

*p < .05 (two-tailed). **/? < -01 (two-tailed). ***/? < -001 (two-tailed). 



Performance measures 



4 5 6 7 8 9 



.35* 

-.01 .42** 



.24 


-.15 


-.20 


- 






.16 


-.18 


-.23 




- 




.23 


-.14 


-.19 


gy*** 


57 *** 


- 


.12 


-.12 


-.12 


^ 




.25 




Was there an effect of type of practice on knowledge of FAST exam procedures and 

FAST exam performance? 

To address this question, we examined whether there were treatment effects on 
knowledge and performance outcomes. We checked for differences on knowledge of basic 
FAST procedures, anatomical interpretation of FAST windows, identification of FAST 
window quadrants, and diagnostic interpretation of FAST windows. Performance was 
examined by checking for differences on time to scan, and the quality of window acquisition 
and window interpretation for the RUQ, LUQ, and suprapubic quadrants. 

Effect of Instruction and Practice on Learning 

Because we recruited novices for this study, we conducted pretest-posttest analyses on 
the knowledge measures to check whether participants learned from the instruction and 
practice. Separate paired t tests were conducted on each knowledge measure as shown in 
Table 5. Participants learned over instruction and practice on all knowledge measures, with 
percent gains of 121%, 133%, 474%, and 535% on basic FAST scanning procedures, 
anatomical interpretation of FAST windows, identification of FAST window quadrants, and 
diagnostic interpretation of FAST windows, respectively. 



Table 5 

Pretest and Posttest Descriptive Statistics and Paired t Tests (N = 49) 





Pretest 


Posttest 


Paired t test 


M 


SD 


M 


SD 


t 


P 


effect size 


Basic FAST scanning procedures 


6.49 


3.03 


14.33 


1.73 


20.64 


<.001 


3.21 


Anatomical interpretation of FAST windows 


5.38 


2.68 


12.51 


2.03 


16.84 


<.001 


3.39 


Identification of FAST window quadrants 


1.53 


2.87 


8.78 


2.47 


17.13 


<.001 


3.24 


Diagnostic interpretation of FAST windows 


1.42 


2.41 


9.02 


1.96 


18.43 


<.001 


4.26 



Effects of Type of Practice on Knowledge 

Separate analysis of covariance (ANCOVA) procedures were conducted on the posttest 
knowledge scales using the pretest score as a covariate (Table 6). There was a significant 
effect of condition on diagnostic interpretation of FAST windows. After controlling for 
pretest scores, participants in the experimental condition (Adj. M = 9.80, SE = 0.40) scored 
significantly higher than participants in the control condition (Adj. M = 8.34, SE = 0.38), E( 1 , 
42) = 6.88, p = .01, d = 0.81. Participants who received simulator-based practice scored about 
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18% higher on items requiring diagnostic interpretation than participants who received 
classroom-based practice. No other differences were found. 



Table 6 

Adjusted Means, Standard Errors, and Analysis of Covariance (ANCOVA) for Posttest Knowledge Measures 









Comparison of treatment effects 










Control 




Experimental 


ANCOVA a 


Knowledge of FAST exam procedure 


n 


Adj. 

M 


SE 


n 


Adj. 

M 


SE 


F ratio 


P 


effect 

size 


Basic FAST scanning procedures 


24 


14.15 


0.30 


21 


14.54 


0.32 


1.63 


.39 


- 


Anatomical interpretation of FAST window 


24 


12.74 


0.41 


21 


12.25 


0.43 


0.66 


.42 


- 


Identification of FAST window quadrant 


24 


8.42 


0.46 


21 


9.19 


0.49 


1.35 


.25 


- 


Diagnostic interpretation of FAST window 


24 


8.34 


0.38 


21 


9.80 


0.40 


6.88 


.01 


0.81 



a Between-groups df- 1, pretest used as covariate. 



Effects of Type of Practice on Performance 

Performance during the live patient exam was evaluated using three measures: (a) time- 
to-scan; (b) quality of the acquired window; and (c) quality of the interpretation of the 
window. Data were analyzed by type of patient and quadrant. 

Time-to-scan. Separate analyses were conducted by type of patient (normal, positive) 
and for each quadrant. For each quadrant, a repeated-measures ANOVA was conducted, with 
type of patient (normal, abnormal) the within-subjects factor, and condition (control, 
experimental) the between- subjects factor. For the RUQ, an effect of type of patient was 
found, with participants in both conditions taking significantly longer to scan the positive 
patient (M = 79.2s, SD = 58.9s) compared to the normal patient (M = 55.8s, SD = 63.7s), F(l, 
48) = 5.78, p = .02, d = 0.38. Participants in general took about 42% longer to scan the 
positive patient. No other differences were found. 

For the suprapubic quadrant, an effect of type of patient was found, with participants in 
both conditions taking significantly longer to scan the positive patient (M = 45.4s, SD = 
48.9s) compared to the normal patient (M = 33.5s, SD = 37.2s), F(l, 46) = 6.16, p = .02, d = 
0.27. Participants in general took about 36% longer to scan the positive patient. No other 
differences were found. 

For the LUQ, a significant main effect was found due to condition, with participants in 
the control condition taking less time to scan the patients (M = 80.4s, SD = 16.9s) than 
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participants in the experimental condition (M = 140.4s, SD = 16.5s), F( 1, 43) = 6.45, p = .02, 
d = 3.59. Participants who received simulator-based practice took about 75% longer to scan 
the patient compared to participants who received classroom-based practice. No other 
differences were found. 

These results suggest differences in the time to scan between the normal and positive 
patients in general. The only effect of practice was on the LUQ quadrant, with participants 
receiving hands-on classroom-based practice performing the scan faster than patients who 
received only simulator-based practice (which did not include practice finding the initial 
anatomical landmarks). 

Window quality. Table 7 shows the distribution of control and experimental 
participants who acquired high-quality windows. High quality was defined as a window 
quality rating of excellent. Other was defined as a window quality rating of fair, poor, or 
other. 



Table 7 

Comparison of Window Quality by Condition (N = 49) 





Control 




Experimental 




Quadrant 


No. of participants with 
high-quality windows a 


Other” 


No. of participants with 
high-quality windows a 


Other b 


Normal patient 


RUQ* 


18 


6 


11 


14 


LUQ 


8 


16 


4 


21 


Suprapubic 


19 


5 


15 


10 


Positive patient 


RUQ 


10 


14 


8 


17 


LUQ 


9 


15 


6 


19 


Suprapubic*** 


16 


8 


4 


21 



a Window quality rating of excellent. b Window quality ratings of fair, poor, or other. 
*p < .05 (two-tailed). ***p < .001 (two-tailed). 



Separate chi-square tests were conducted for each quadrant by type of patient. For the 
normal patient (RUQ), the number of high-quality windows acquired was greater than 
expected in the control condition, and the number of non-high-quality windows was greater 
than expected in the simulation condition (tp = -.32, p = .03). Seventy-five percent of 
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participants who received classroom-based practice were able to acquire high-quality 
windows, compared to 44% of participants who received simulator-based practice. 

Similarly, for the positive patient (suprapubic quadrant), the number of high-quality 
windows acquired was greater than expected in the control condition, and the number of non- 
high-quality windows was greater than expected in the simulation condition (cp = -.52, p < 
.001). Sixty-seven percent of participants who received classroom-based practice were able 
to acquire high-quality windows, compared to 16% of participants who received simulator- 
based practice. 

Window interpretation. Table 8 shows the distribution of control and experimental 
participants who correctly interpreted windows. Other was defined as a rating of incorrect or 
other. 



Table 8 



Comparison of Window Interpretation by Condition (N = 49) 





Control 




Experimental 




Quadrant 


No. of participants 
with correct 
interpretations 


Other a 


No. of participants 
with correct 
interpretations 


Other a 


Normal patient 


RUQ 


24 


0 


24 


1 


LUQ* 


17 


7 


10 


15 


Suprapubic 


18 


6 


23 


2 


Positive patient 


RUQ 


18 


6 


17 


8 


LUQ 


11 


13 


14 


11 


Suprapubic 


11 


13 


7 


18 



“Rating of incorrect or other. 
*p < .05 (two-tailed). 



Separate chi-square tests were conducted for each quadrant by type of patient. For the 
normal patient (LUQ), the number of windows interpreted correctly was greater than 
expected in the control condition, and the number of windows interpreted incorrectly was 
greater than expected in the simulation condition (cp = -.31, p = .03). Seventy-one percent of 
the participants who received classroom-based practice were able to interpret windows 
correctly, compared to 40% of the participants who received simulator-based practice. 
Conditional analyses were also conducted to examine whether there was a difference in 
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diagnosis quality given an adequate window acquisition (i.e., a window rating of excellent or 
fair). There were no differences between conditions by quadrant and patient, or by overall 
diagnosis quality. 

Summary of performance results. Table 9 summarizes the performance results. In 
general, there were no condition differences in scan time for RUQ and suprapubic quadrants, 
but the LUQ participants who received hands-on classroom training completed the scans 
faster. When window quality is examined, the control condition had more participants 
acquiring excellent window scans compared to the experimental condition, but only for two 
out of the six scans. Similarly, when window interpretation is examined, the control 
condition had more participants rendering a correct diagnosis compared to the experimental 
condition, but only for one out of the six scans. 

The performance results suggest that there is little statistical difference in acquisition 
time between the control and experimental conditions, except for LUQ. Hands-on classroom 
practice appears to influence window acquisition and window interpretation only under very 
specific conditions: RUQ normal and suprapubic positive for window quality, and LUQ 
normal for window interpretation. In terms of diagnostic interpretation of acquired window 
scans, particularly for the positive patients, there was no statistical difference between 
conditions. The only difference found was in the interpretation of the LUQ scan for the 
normal patient; in this case, participants who received classroom-based practice were more 
successful at correctly interpreting the scans than participants who received simulator-based 
practice. 



21 




Table 9 



Summary of Performance Differences 



Performance 

measure 




FAST quadrant 




RUQ 


LUQ 


Suprapubic 


Time to scan 


No effect of condition: 
Positive patient takes 
longer to scan than normal 
patient. 


No effect of patient: 
Control condition takes 
less time to scan than 
experimental condition. 


No effect of condition: 
Positive patient takes 
longer to scan than normal 
patient. 


Window quality 








Normal patient 


control > experimental 


No effect of condition 


No effect of condition 


Positive patient 


No effect of condition 


No effect of condition 


control > experimental 


Window 

interpretation 








Normal patient 


No effect of condition 


control > experimental 


No effect of condition 


Positive patient 


No effect of condition 


No effect of condition 


No effect of condition 



Summary 

These results suggest that there was an impact of the type of practice on participants’ 
knowledge of disease states given a FAST window, favoring the simulator-based practice. 
Hands-on practice in the classroom training setting appeared to provide participants with 
probe manipulation skills only on the most complex scan (LUQ), resulting in faster window 
acquisition times than participants in the experimental condition. Both types of practice 
appeared to result in similar FAST exam performance levels with respect to window quality 
(LUQ) and interpretation (RUQ, suprapubic). Where performance differences existed, the 
pattern of difference was unclear, varying by type of measure, quadrant, and type of patient. 

Participants’ Perceptions of the Effectiveness of Practice 

In this section we examine participants’ perceptions of the utility and effectiveness of 
the practice received. 

Survey Responses 

Participants were asked to evaluate how effective the practice session was for preparing 
them for the live patient exam. Participants were asked “How well did the ultrasound practice 
session (either with the simulator or in the group session) prepare you to perform a ‘real’ 
ultrasound examination of a live patient?” The distribution of participant responses is given 
in Table 10. Because participants’ responses were clustered around the two middle 
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categories, responses were collapsed into two categories: inadequate (representing the two 
scale points of very inadequate and inadequate amount of practice) and adequate 
(representing the two scale points of adequate and too much practice). Chi-square tests were 
conducted for each question, to test for an association between condition and participants’ 
perceptions of the adequacy of the amount of practice. 

Table 10 



Distribution of Participant Responses About the Amount of Practice 





Very inadequate 
amount of 
practice 


Inadequate 
amount of 
practice 


Adequate 
amount of 
practice 


Too 

much 

practice 


Physical aspect of scanning (i.e., manipulating 
the probe) 










Control 


0 


6 


17 


0 


Experimental 


0 


14 


11 


0 


Acquisition of a scan window aspect of 
scanning (i.e., being able to acquire a high 
quality window) 










Control 


1 


11 


12 


0 


Experimental 


1 


8 


16 


0 


Diagnostic aspect of scanning (i.e., being able 
to identify normal or abnormal conditions) 










Control 


2 


15 


6 


0 


Experimental 


2 


10 


13 


0 



For the physical aspect of scanning, the number of participants reporting adequate 
amount of practice was greater than expected in the control condition and the number of 
participants reporting inadequate amount of practice was greater than expected in the 
experimental condition (cp = -.30, p = .04). For the acquisition aspect of scanning, there were 
no differences by condition. For the diagnostic aspect of scanning, a potential effect was 
found. The number of participants reporting adequate amount of practice was greater than 
expected in the experimental condition and the number of participants reporting inadequate 
amount of practice was greater than expected in the classroom condition (cp = -.27, p = .07). 

Participants in both conditions appeared to perceive the adequacy of the practice 
sessions differently. Participants who received classroom-based practice reported adequate 
practice (in general) with respect to acquiring a window compared to participants who 
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received simulator-based practice, and inadequate practice with respect to diagnosing 
window scans compared to participants who received simulator-based practice. 

Written Comments 

An examination of participants’ written comments on the preparedness for diagnosing a 
live patient suggests that participants in the control condition desired more practice with both 
normal (for a better grasp on anatomic variation) and abnormal (for a range of pathology and 
severity) cases. Participants in the experimental condition indicated that the simulator 
practice helped them recognize the variations in both normal and abnormal cases, and in 
interpreting those images. 

Overall, participants in the control condition commented that both instruction on how 
to handle the probe and actual “probe time” were beneficial in learning the physical aspect of 
scanning. These participants also noted that the use of a single patient in the practice session 
made it difficult for them to adjust their performance on different patients. Participants in the 
experimental condition noted that practice with the simulator probe familiarized them with 
image orientation, probe movement, and visualizing major structures. The primary difficulty 
for these participants was locating anatomical landmarks. Both range of movement and the 
initial (correct) placement of the probe on the simulator model were set. Participants 
commented that practice placing the probe on the virtual patient would be helpful in 
developing skills in locating landmarks on a live patient. 

Participants in the control condition reported that adequate window acquisition was 
dependent on the window being viewed (the LUQ was harder to acquire) and on the patient 
(easy on practice patient, difficult on live exam patient). Experimental condition participants 
noted that adequate window acquisition on a live test patient was dependent on being able to 
locate the proper anatomical landmarks and place the probe in the correct initial position. 
Participants commented that practice on the simulator increased confidence in finding and 
determining a good scan window once the probe was already in place. 

Overall, participants viewed both the video instruction and practice sessions as helpful 
in answering posttest questions on window identification and interpretation. Participants in 
the experimental condition commented that the video instruction was useful for basic 
understanding but simulator practice was key in aiding spatial understanding, identification 
of visual patterns of structures, and increased confidence in window interpretation. 

When asked what would improve performance while conducting the FAST exam, 
experimental condition participants perceived the simulator to be an effective means of 
ultrasound training but also responded that expert instruction (especially with regard to 
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placement of the probe, amount of pressure to exert on a live patient), additional simulator 
practice time, and practice with a live patient were desired. 

Participants’ Perceptions of the Effectiveness of the FAST Simulator 
(Experimental Condition Only) 

In the last two data collection waves we inserted several questions into the protocol for 
participants receiving simulation practice. Participants in the first data collection wave had 
commented to us how useful the simulation practice was. The new questions were designed 
to gather participants’ perceptions of what the simulation practice helped them with. Table 11 
shows the distribution of participants’ comments. In general, there was general agreement 
among participants that the simulation practice helped them with window interpretation, and 
less agreement with respect to window acquisition. 

Table 11 



Distribution of Simulation Participant Comments 



Question 


Strongly 

disagree 


Disagree 


Agree 


Strongly 

agree 


With respect to the performance test 










Practicing with the simulator prepared me to interpret the 
scans during the live patient test 


0 


1 


12 


6 


Practicing with the simulator prepared me to acquire the 
windows during the live patient test 

With respect to window interpretation 


1 


6 


9 


3 


The simulator was effective in helping me interpret scans 
of different kinds of conditions 


1 


3 


12 


3 


The simulator was effective in helping me interpret scans 
of the different quadrants 


0 


2 


13 


4 



When asked to check one of four options reflecting how confident participants were 
about performing a FAST examination with actual patients, one participant selected with no 
further training ; 10 participants selected with a few practice scans on a person', three 
participants selected with a training class on ultrasound scanning', and three participants 
selected other. 

Written Comments 

We also asked participants for written comments about the most useful features of the 
simulator. In general, participants’ comments can be grouped into the following descriptions: 
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• The capability for real-time visualization, positioning, and manipulation of the 
probe 

• The capability to compare their window to the four reference scans (normal, and 
three abnormal windows) 

• The instructor evaluation of the diagnosis and quality of scan (although this is an 
artifact of the study procedures) 

• The capability to compare windows of normal and abnormal conditions 

• Practice via multiple cases 

• The layers or skeleton overlay 

We also asked for comments on what improvements could be made to the simulator. In 
general, participants’ comments can be grouped into the following descriptions: 

• A fixed initial probe position is helpful for beginners, but over time, the user should 
be responsible for positioning the transducer by locating appropriate anatomical 
landmarks on the body. 

• Allow for lateral (superior/anterior) movement of the probe, rather than just 
rotational movement; expand range of motion of the probe. 

• Expand the field of view of the ultrasound scans. 

• In addition to the range of normal and abnormal images displayed after window 
selection and diagnosis, also display a “best window” image which also outlines the 
organs and abnormality. 



DISCUSSION 
Summary of Findings 

Participants learned from the training materials. Evidence of the effectiveness of 
the training materials (the instructional videos and the practice) is seen by the significant and 
large gains on all knowledge-based measures. Participants more than doubled their scores on 
the posttest, with percent gains of 121%, 133%, 474%, and 535% on basic FAST scanning 
procedures, anatomical interpretation of FAST windows, identification of FAST window 
quadrants, and diagnostic interpretation of FAST windows, respectively. How much each 
training component (i.e., the instructional videos vs. the practice) contributed to the posttest 
gains cannot be determined because measures were not taken between the instruction and 
practice components of the study. 

Simulator-based practice promotes greater acquisition of window interpretation 
knowledge (i.e., on the paper-and pencil measure) compared to classroom-based 
practice. There was clear evidence that participants in the simulator practice condition were 
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able to interpret correctly more FAST windows than participants who received classroom 
practice (18% higher scores on the knowledge posttest). In general, participants in the 
simulator condition practiced on multiple normal and positive cases (vs. a single normal 
case), and were able to compare an acquired window to reference windows that represented 
normal and positive states (minimal, mild, severe). Participants in the simulator practice 
condition reported overwhelmingly that the simulation practice aided them in interpreting 
scans. In addition (although marginally significant, p = .07), the number of participants 
reporting adequate amount of practice was greater than expected in the simulator practice 
condition and the number of participants reporting inadequate amount of practice was greater 
than expected in the classroom practice condition. This finding points to a major benefit of 
the FAST simulator-based practice: The capability to offer practice with multiple cases and 
conditions synchronized with the probe manipulation. 

Simulator-based practice appears to be similar to classroom-based practice in 
diagnostic accuracy of windows acquired during live patient exams. The pattern of 
results on window interpretation during the performance test with model patients suggests no 
difference between practice conditions. That is, across two patient types (normal, positive), 
there was no difference on five of the six quadrants in the number of participants who made 
correct interpretations. The exception was for the LUQ (normal patient). In this case, more 
classroom practice participants were able to correctly diagnose the window. 

Classroom-based practice may promote greater window acquisition skills 
compared to simulator-based practice. The pattern of results on the acquisition of quality 
windows is less clear. Acquisition of high-quality windows appeared the most challenging 
for the LUQ, followed by the RUQ and suprapubic quadrant. Scan time data are consistent 
with this ordering, with participants taking over 2 minutes to scan the LUQ, under 2 minutes 
for RUQ, and little over a minute for the suprapubic quadrant. Statistical differences in 
window quality were found that favored the classroom-practice condition for the RUQ 
(normal patient) and suprapubic quadrant (positive). In both cases, a higher number of 
participants in the classroom-practice condition were able to acquire excellent windows 
compared to the simulator-practice condition. 3 

These results may be pointing to one important difference between practice conditions. 
In the classroom-practice condition, participants received individualized guidance from the 
instructor on probe positioning and rotation, and sometimes included the instructor physically 



3 While not statistically significant, window quality favored the classroom-practice condition on all other 
quadrants/patients. 
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guiding the participant’s hand to establish correct placement on the patient. In contrast, the 
simulator had the probe locked into the ideal initial positions. This difference was a major 
issue that surfaced in participants’ self-reports. For instance, participants in the classroom- 
practice condition reported that they received an adequate amount of practice compared to 
participants in the simulator-practice condition, who reported an inadequate amount of 
practice. Simulation-practice participants’ written comments also reflected their desire for 
physical practice, with one participant’s comments summarizing the issue: 

I think it would have been more helpful if we had practice placing the probe onto the 

patient instead of having it already in place and only being able to change the angle. 

When I did the procedure on the patient, it was hard for me to know where to put the 

probe. 

There appears to be no difference in the time to scan a quadrant between the two 
practice formats. For the RUQ and suprapubic quadrant, there was no effect of condition: 
Participants in general took significantly longer to scan the positive patient than the normal 
patient (RUQ: 42% longer; suprapubic: 36% longer). For the LUQ, there was an effect of 
condition but not an effect of patient, with the simulation-practice condition taking 
significantly longer to scan the patients (75% longer). While there were no statistically 
different scan times due to condition on two of the three quadrants, these quadrants had large 
standard deviations in general, with the control condition having significantly larger scan 
time standard deviations. 

Overall, these findings suggest the general effectiveness of the simulator practice: 
(a) superior diagnostic interpretation on a knowledge-based test; (b) potentially similar levels 
of performance on live patient exams — despite having no prior hands-on practice (i.e., the 
first scan with a patient was the live patient test); and (c) positive perception by participants 
and recognized utility of the simulator. 

Limitations 

There are three limitations to this study. First, the number of model patients used for 
the live test was limited to two for practical reasons. The low number of patients limited the 
range of cases and situations a participant was tested on. Thus, open questions include 
sampling (Do the model patients adequately represent the patients likely to be encountered 
under actual conditions? Are there peculiar features of the model patients that would bias the 
performance in some way?) and generalizability — to what extent does performance on the 
live test in this study adequately represent performance on actual patients likely to be 
encountered? 
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A second limitation is in the study procedures. Because of the limited availability of 
ultrasound machines for live patient testing (only two), the extremely limited pool of patients 
with ascites willing to participate in this study, and the blocking of participants (i.e., the 
classroom-practice participants), the procedure was designed to maximize participant 
throughput and minimize patient discomfort (how long the patient was being scanned). This 
resulted in suboptimal counterbalancing across model patients during the live patient testing 
(participants were assigned to patients based on patient availability which generally 
preserved counterbalancing but did not guarantee it), randomization at the time of scheduling 
the appointment for the study (vs. randomization at the time of arrival, which would mitigate 
against potential biases resulting from schedule availability such as only medical students in 
cohort X were available because the other cohorts had class), potential changes in the model 
patient over time resulting in blocks of participants receiving different patient conditions 
(e.g., if patients had to go to the restroom in the middle of the three-hour testing window, 
then their bladder would be empty for a while, making suprapubic window acquisition 
harder). We speculate that the latter situation may be a possible explanation for the advantage 
found for the control condition on window quality, as the control condition participants 
generally started the live patient exams before the experimental condition patients. 

The third limitation is a potential training effect in the classroom-practice condition. 
We do not have information on how representative the classroom instructor was of 
ultrasound trainers in general. The instructor used in this study reported over 20 years of 
experience teaching ultrasound concepts and procedures. Our observation of the classroom 
instruction suggested an instructor who was able to provide clear explanations and 
demonstrations of procedures, and provide effective feedback and guidance to students who 
had difficulty acquiring a window. In addition, the class size used (about eight participants) 
may have been smaller than a typical training class. 

Implications for FAST Training 

One of the most interesting findings of this study was that using the virtual trainer 
(Sonosimulator™) for practice did not result in markedly inferior performance on the 
physical aspects of scanning. That is, the training (the instructional videos and the simulator- 
based practice) was sufficiently effective to enable participants who received no hands-on 
practice to perform comparably to the participants who received hands-on practice on most 
of the performance measures across the normal and positive patients used in this study. 

However, simulator-practice participants’ self-reports and a general trend favoring 
classroom-practice (but not statistically significant) on window acquisition, point to the 
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importance of being able to find the initial probe location for a quadrant. While in this study 
participants in the simulator-practice condition did not receive such practice (because of the 
technical complexity of the modeling), two solutions exists that are simple, straightforward, 
and practical. Hands-on training can be implemented in a few minutes per quadrant based on 
our observations of the classroom training, or a multimedia demonstration of initial probe 
placement could be used to mimic the hands-on training. 

In the broader training context, the findings of this study are consistent with findings of 
design features of effective simulators. The simulator used in this study was designed around 
the cognitive demands of FAST window acquisition and interpretation. Through repeated 
exposure to cases, users are engaged in the review, identification, acquisition, and 
interpretation of a FAST window. High fidelity is used judiciously — only to link the real- 
time probe response to its corresponding window. Users can view interpolated windows of 
scans of actual patients with varying free fluid conditions and varying anatomy, and then 
compare an acquired window to windows of normal and various positive conditions. This 
latter capability is an important instructional feature as it helps users identify the window 
characteristics of the various free fluid states. 

One of the most important training benefits of simulators is extended training time. 
Having available virtual patients to scan with various disease conditions avoids the 
restrictions associated with model patients, such as availability and willingness to endure 
long training sessions with a number of trainees. In the case of ascites, patients are often too 
ill to even participate in a study. Another benefit of the simulator is documented pathology — 
any anatomical anomalies or a specific severity of condition can be included in the simulator 
for an unlimited number of views. 

Finally, the utility of the simulator as an anytime- anywhere refresher trainer is clear. 
Assuming basic competency at initial probe location and landmark identification, the use of 
the FAST simulator as a means to relearn procedures anytime-anywhere seems ideal as the 
simulator emphasizes the mapping between probe movement and the window quality. 
Perhaps the most powerful capability of the simulator is to provide users with practice 
recognizing various disease conditions or window anomalies (e.g., artifacts) that would be 
difficult to observe otherwise. 

Future Directions 

Simulator Design 

One future direction for the SonoSimulator™ is to include the capability for users to 
practice finding the appropriate initial location for the probe. While this may be ideal, this 
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capability may be cost prohibitive. A second direction for the SonoSimulator™ system may 
be using the image database to diagnose live scans. Given a particular quadrant, a given 
window image could be compared to the SonoSimulator™ image database (a volume with 
known anatomical and disease properties) and a similarity measure computed. 

Follow-up Studies 

One follow-up study would be to examine the relative lift in performance due to the 
simulator. That is, vary the sequence of simulator and classroom practice where in one 
condition, classroom practice precedes simulator practice, and vice versa for the second 
condition. This study would address the practical issue of whether to use the virtual simulator 
as part of a FAST training package. Alternative designs could vary degree of guidance from 
classroom practice as done in the current study to minimal guidance where the hands-on 
practice focuses on probe placement. Skill retention studies could also be conducted to 
examine skill decay of window acquisition and window interpretation over time. 

Another study could focus on optimizing window interpretation. Given the database of 
patients with different diseases and quadrants, one training study could examine whether it 
would be more efficient to separate the exposure to different abnormal conditions from the 
simulator scanning activity — can window interpretation be bolstered independent of the 
simulator? A third study could examine the extent to which the simulator can be used as an 
assessment of FAST window acquisition and interpretation. Given the variation in live 
patients, the cost involved, and the availability of patients with different diseases, the use of a 
FAST simulator with a large database of cases could standardize the administration and 
content of FAST exams. 
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National Center for Research 

on Evaluation, Standards, & Student Testing 

UCLA Graduate School of Education & Information Studies 



Ultrasound Simulator Efficacy Study 
Summer 2010 



Ultrasound Training Posttest 



DIRECTIONS 

You will be asked to: 

1. Answer questions on anatomy and ultrasound procedures, and ultrasound scans. 

2. Fill out a survey that asks about demographic information and experience with 
ultrasound scanning procedures. 

3. If you do not know the answer to any question, write don’t know , DK, or ?. 

Thank you for helping us with this study. 
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1. Dale: 

2. Time now: 



For each question, please circle the best answer. If you do not know the answer, 
select “don’t know* 55 



3. When an organ is described as hypoechoic in relation to another organ it means 

a. the organ is brighter relative to the first organ. 

b. the organ is less echogcnic relative to the first organ. 

c. the organ is visualized inferior to the first organ. 

d. the organ is visualized posterior to the first organ. 

e. don't know 

4. The sonographic appearance of the normal renal cortex is 

a. hyperechoie relative to the liver, 

b. heterogeneous. 

c. hypoechoic relative to the liver. 

d. highly reflective. 

e. don't know 

5. The lie of the I VC (Inferior Vena Cava) within the body . 

a. makes it impossible to visualize with ultrasound, 

b. is superior to inferior. 

c. is oblique. 

d. is ventral to the aorta. 

e. don't know 

6. Typically, when scanning in a coronal plane, . 

a. the indicator on the transducer is aimed toward the patient' s left. 

b. the indicator on the transducer is aimed toward the patient's right. 

c. the ultrasound beam enters the body from a ventral to dorsal direction. 

d. the ultrasound beam can enter the body in either a right or left lateral direction. 

e. don't know 
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7. When scanning in a transverse plane, the ultrasound beam enters the body from 

a. any of the following: an anterior, posterior, right lateral, or left lateral direction. 

b. only an anterior or posterior direction. 

c. only a right or left lateral direction. 

d. either an anterior, posterior, or medial direction. 

e. don’t know 

8. When scanning in a sagittal plane, the ultrasound beam enters the body from 

a. either an anterior or posterior direction. 

b. any of the following: an anterior, posterior, right lateral, or left lateral direction. 

c. either an anterior, posterior, or medial direction. 

d. either a right or left lateral direction. 

e. don’t know 

9. Free fluid in the abdomen is most commonly seen . 

a. in Morrison’s Pouch. 

b. behind the kidneys. 

c. under the diaphragm. 

d. in the Pouch of Douglas. 

e. don’t know 

10. All of the following are true regarding free fluid in the abdomen, EXCEPT: 

a. It is black. 

b. It fills potential spaces. 

c. It is hyperechoic. 

d. It is anechoic. 

e. don’t know 



Please fill in the blanks for the questions below. Write if you do not know the 

answer. 



1 1. When imaging the patient in a sagittal plane, aim the indicator on the transducer towards the 
patient’s . 



12. When imaging the patient in a transverse plane, aim the indicator on the transducer towards the 
patient’s . 
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For the questions below, please label the anatomy of the images shown. Write “?” if 
you do not know the answer. 
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Please match the scans images on the left to the probe position used to acquire the 
scan. Write 44 ?” if you do not know the answer. 



15 . 
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Please match the anatomical representations on the lacing page to their ultrasound 
scan counterparts on this page. Write **?” if you do not know the answer. 



16 . 
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| Please mark the appropriate boxes in both columns. 



17. 




Nonna 1 (Negative) 


□ 


RUQ 


Abnormal (Positive) 


□ 


LUQ 


Inadequate to interpret 


□ 


Sub-Xiphoid 


Don’t know 


□ 

□ 


Suprapubic 
Don’t know 



18 . 




□ Nonna] (Negative) 

□ Abn orma I s Posit i ve ) 

□ Inadequate to interpret 

□ Don’t know 




□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don’t know 



□ RUQ 

□ LUQ 

□ Sub -Xiphoid 

□ Suprapubic 

□ Don’t know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don’t know 



UltrastHjndPosttesM0D715.doc^ Page 11 of 16 

©2010 The Regents of the University ot California 

National Center for Research on Evaluation, Standards, and Student Testing 



46 




ID: 1015 



20 . 



21 . 



22 . 




□ Normal (Negative) 

□ Abnormal (Positive) 

P I n adequate to i nte rp ret 

□ Don't know 



□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don’t know 




□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don't know 




□ RUQ 

□ LUQ 

Q Sub-Xiphoid 

□ Suprapubic 

□ Don’t know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don't know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don’t know 

□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don't know 
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24 . 



26 . 







□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don't know 



□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don't know 



□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don't know 



□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don't know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don't know 

□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don't know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don't know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don't know 
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2K. 




□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don’t know 



29 . 




□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don’t know 




□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don’t know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don’t know 



□ RUQ 

□ LUQ 

□ Sub-Xiphoid 

□ Suprapubic 

□ Don’t know 



31 . 




□ Normal (Negative) 

□ Abnormal (Positive) 

□ Inadequate to interpret 

□ Don’t know 



□ RUQ 

□ LUQ 

□ Sub- Xiphoid 

□ Suprapubic 

□ Don’t know 
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| Please identify the type of ultrasound scanning artifact shown in the image" 



32. 




□ High attenuation 

□ Low attenuation 

□ Refraction 

□ Mirror Image 

□ Reverberation 

□ Don’t know 



.O" I □ High attenuation 

I □ Low attenuation 
I □Refraction 

* \ I □ Rib Shadow, Image 

J-' L I □Don’t know 



34. Time now: 
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BACKGROUND 

35. Age: years 

36. Gender: □ Male □ Female 

37. Current position: 

□ Medical student 

□ Medical resident 

□ Physician 

□ Nursing student 

□ Other 

38. Specialty: 

39. MC AT Score: □ N/A 

40. Nol including today, please describe your experience with ultrasound scanning procedures. Check 
all that apply. 

□ No training in ultrasound scanning procedures 

□ Lecture / classroom training. Date: 

□ Hands-on practice. Date: 

□ Other: 

Please describe;^ Date: _ 

41. Nol including today, how many hours of training on ultrasound procedures have you had? 

□ 0 hours 

□ 0 - 2 hours 

□ 2 — 4 hours 

□ 6 8 hours 

□ 8 — 10 hours 

□ More than 10 hours 
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Would you like to participate in this study as one of the model patients (the person getting scanned)? If so, please 
provide your contact information. 



Would you like to participate in any future training studies? If so, please provide your contact information. 



THIS IS THE END OF THE STUDY. 
Please hand your booklet to the researcher. 
Thank you very much for participating. 
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APPENDIX B: 



PRIOR KNOWLEDGE OF ANATOMY ITEMS 
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APPENDIX C: 



PARTICIPANT FEEDBACK FORM 
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Feedback on the Practice Session 



ID; 



We would like to get your lee a back on the ultrasound scanning practice you received in 
this study. By practice, we mean either the session where you and other trainees 
practiced scanning on a live patient, or the simulation practice where you used a 
computer to practice scanning a virtual patient. 

We would appreciate your honest and candid comments — positive or negative 

1 How well did the ultrasound practice session (either with the simulator or in the 
group session) prepare you to perform a "rear ultrasound examination of a live 
patient? 

Only consider the physical aspect of scanning (i.e manipulating the probe) 

□ Very inadequate amount □ tnad equate amount □ Adequate □ Too much practice 

of practice of practice amount of 

practice 

Please explain 



Only consider the acquisition of a scan window aspect of scanning {i.e. being 
able to acquire a high quality window) 

□ Very inadequate amount □ inadequate amount □ Adequate □Too much practice 

of practice of pra ctice a mount of 

practice 

Please explain. 



Tr» jj"ft ■ 1 0 07 1 6 *)« Plot 2 of a 

■£'2010 'Hia Etoganb of r* U'nrtrsny afCa Heims 

Notorial Cent' Re sea rah -zn Evasion, Standards. and Budent Te^ig 
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Only consider the diagnostic aspect of scanning (i.e. being able to identify normal 
or abnormal conditions) 

□ Very inadequate amount □ Inadequate amount □ Adequate □ Too much practice 

of practice of practice amount of 

practice 

Please explain. 



2. For questions like the one shown below on the posttest. 




■J Nr/' mild [Ncjpalivc i 

□ AtHHTIBll(PPflUve) 
J limdeqiulc la inteqtrcl 
J UiJl'TbKrtv 



J RLTQ 

LI LUQ 

LI 

J $I^H^nb4c 
LJ iXuCi Saiaw 



Which part of the training helped you the most to answer the above type of 
questions: 

□ The □ The practice □ Both contributed - 1 other 

computer session equally 

videos 

Please explain. 
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PLEASE ANSWER THE FOLLOWING QUESTIONS ONLY IF YOU USED 
THE COMPUTER-BASED SIMULATOR 



3. What were the most useful features of the simulator? 



4. How could the simulator be improved? 



5. What would have helped you improve your performance conducting the FAST scan 
on today's patients? 



Indicate how much you AGREE with the following! statements. 


Strongly 

Disagree 


Disagree 


Agree 


Strongly 

Agree 


6. 


Practicing with the simulator prepared me to interpret the 
scans during the live patient test 


1 


2 


3 


4 


7. 


Practicing with the simulator prepared me to acquire the 
windows during the live patient test 


1 


2 


3 


4 


8. 


The simulator was effective in helping me interpret scans of 
different kinds of conditions 


1 


2 


3 


4 


9. 


The simulator was effective in helping me interpret scans of 
the different quadrants 


1 


2 


3 


4 



10. Having gone through today's training, I feel confident that I could perform a FAST 
scan (involving only the RUG : LUQ : and suprapubic quadrants) with actual 
patients... 

□ With no further □ With a few practice □ With a training class on □Other 

training scans on a person ultrasound scanning 



T r a irmg feedbacfc.i607i 6. do ox P age 4 of 3 
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APPENDIX D: 



PARTICIPANT COMMENTS 
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How well did the ultrasound practice session prepare you to perform a “real” 



ultrasound examination of a live patient? Only consider the physical aspect of scanning: 



ID 


Comment 


1002 


It took 2 practice attempts, but I felt confident. 


1003 


The LUQ is harder to visualize and difficult to access when you're short. 


1004 


I think the instruction was great and I knew what I was supposed to do with the probe to obtain a 
good image, but more time personally using the transducer would be helpful. 


1005 


taught to handle probe 


1006 


not enough time to practice ID structures in LUQ & did not get to do it myself 


1008 


Need feedback from instructors on where to place probe & in which direction to rotate 


1009 


some of the images on a real patient (everything but bladder) were very difficult to obtain 


1011 


I know what I was looking for; hard to get right spot, the computer simulation was very helpful for a 
beginner. 


1012 


I had no idea where to initially place probe. What landmarks to use in computer simulator probe was 
already in location just had to move it up/down right/left to get image in better view. 


1013 


I couldn't do an UTZ at all before today & simulator I practice + learned enough to be able to do it. 


1014 


I think it would have been more helpful if we had practice placing the probe onto the patient instead 
of having it already in place and only being able to change the angle. When I did the procedure on 
the patient, it was hard for me to know where to put the probe. 


1017 


It was hard to extrapulate to the exam because the patients were different and we were only able to 
practice on a single person. 


1018 


very helpful 


1021 


no variation of subjects during practice session made it difficult to perform the same test on new 
patient 


1023 


Did not allow for initial approach to find the proper spot at the probe was already in the right spot 
(i.e.LUQ) 


1024 


Good practice, very helpful! 


1025 


I had computer practice and had trouble locating the anatomical landmarks on the patient because I 
had no practice doing this. 


1026 


I wish we had more time with using the probe and better manipulation hands-on teaching. 


1027 


The only other thing would be allowing the student to place the probe on the patient in the simulator, 
instead of having a fixed spot. 


1028 


In our practice session we were never told how to actual hold and manipulate the probe (i.e. no 
feedback from instructor upon review of our answers) 


1029 


I felt it was easier on the simulator. I was very uncomfortable doing it on the patient 


1031 


Nothing is quite like the real thing. There are no borders or artificial transducer constraints in the real 
live scenario and that makes things difficult. 


1032 


Range of manipulation of the probe is too limited in simulator. 


1033 


1 . The simulator provided already correct location of probe placement when placed in front of live 
patients, harder to get correct location. 2. Manipulation of probe had a different feeling on real person 
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ID 


Comment 




vs. computer. 


1035 


I needed a little more time just to play with the machine and get comfortable with it. I improved, but 
I still wouldn't feel comfortable with a real patient. 


1038 


couldn't quite tell abnormal 


1039 


There is no replacement for holding the probe/muscle memory. 


1040 


actual practice » computer tutorial 


1042 


The practice session being simplified in simulator fashion really helped me get my bearings with 
major structures and "which way's up." 


1043 


I'm good at reading scans but not at knowing where to place the actual transducer. 


1045 


It was good practice with orientation. 


1046 


Need someone to answer questions 


1047 


I felt comfortable moving the probe but was not always sure I was at the right height. 


1049 


The probe in the simulator only moves in a fixed axis so it is hard to figure out up & down 
movements, etc. 
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How well did the ultrasound practice session prepare you to perform a “real” 
ultrasound examination of a live patient? Only consider the acquisition of a scan 
window aspect of scanning: 



ID 


Comment 


1001 


some more techniques and tricks could have been emphasized i.e. how to get around the rib shadows 


1002 


It took 2 practice attempts, but I felt confident 


1003 


LUQ hard to visualize 


1004 


This is very dependent on the window I was acquiring. The landmarks for the right upper quadrant 
and suprapubic views were easy to obtain, while left upper quadrant was difficult for me. 


1005 


need smaller group & more hands on 


1006 


should let us experiment with moving the probe more— especially in LUQ since the location of the 
spleen varies 


1008 


did not get to practice changing settings (e.g. gain, depth, etc.) 


1009 


some of the images on a real patient (everything but bladder) were very difficult to obtain 


1011 


The computer simulation made me feel much more confident on what a good window was. 


1012 


The simulator was good at helping me refine this skill. 


1013 


I practiced & learned to improve simulator which allowed me to get better for real person. 


1014 


I think it was helpful to get feedback and redo it if I didn't get a good quality window. 


1017 


same as above 


1018 


very easy on practice patient, hard on test patient 


1023 


allowed feedback to proper window 


1025 


The computer program prepared me well for this. 


1026 


Scanning got easier throughout the process. 


1027 


more time. 


1028 


Ultrasound is a hard skill to learn, and an hour practice with no feedback didn't teach me how to 
adequately obtain a proper window. I just kept on teaching myself either the wrong way to go about it 
(or eventually teaching myself how to acquire window). 


1029 


I found it hard. 


1031 


Its easy to do in the simulation because eventually if you move your transducer around enough you'll 
find a window that looks like the examples. On the real patient I didn't even know where exactly to 
place the transducer. 


1032 


The feedback received during the simulation was useful. 


1033 


same reasons as above 


1035 


This was tougher than it looks. I definitely need more practice. 


1036 


the depth/gain setting were set for us. . . we didn't have to set them ourselves 


1037 


Didn't actually get to turn nobs and adjust window myself 


1040 


Takes physical practice to acquire a handle on the probe to get a feel for the slight movements in hand 
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ID 


Comment 




motion w.r.t. what appears on the screen. 


1042 


I knew generally what I was looking for after the simulator and how to move the transducer, however, 
it was much more challenging on a real patient! 


1043 


It's luck. If I find the major landmark, I can do it well. Finding the major landmark is hard. 


1044 


On the practice session, the probe was already placed at a position where it was only able to rotate & 
not move much. On a real patient, I would be unsure of where to place the probe if it were the 1st 
time. 


1046 


Video goes through it very quickly 


1047 


I was confident that I could find an appropriate scan window if I found the right location. 


1049 


Same as above, other movements simulator doesn't allow. Also how hard to press, etc. 
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How well did the ultrasound practice session prepare you to perform a “real” 
ultrasound examination of a live patient? Only consider the diagnostic aspect of 
scanning: 



ID 


Comment 


1001 


would have been nice to practice on an abnormal or multiple normals first to get a sense of anatomic 
variation 


1002 


No abnormal findings during practice sessions. 


1003 


We don't really know what normal is, just that we're looking for anechoic 


1004 


Again, just because we didn't practice finding abnormal. On the other hand, I think the tutorials did a 
good job preparing us to know what to look for. 


1005 


1st time seeing it. Just learned what normal looks like-but it can vary. 


1006 


Difficulty w/LUQ in different patients 


1007 


More explanations of abnormal results would have been helpful. 


1008 


sometimes can't tell difference between artifact & actual pathology 


1009 


I still feel I need practice identifying 1. anatomy and 2. normal v. abnormal 


1011 


I thought I did pretty well, but of course, there are so many abnormalities I still don't know. But for 
what was taught, I thought I got it. 


1012 


Computer simulator allowed me the opportunity to see many variations of normal and abnormal 
which I believe is key to becoming comfortable with interpreting images and making diagnosis. 


1013 


I practiced & learned to improve simulator which allowed me to get better for real person. 


1014 


It would have been more helpful if we had more "abnormal" sample images to look at. Also, it would 
have been nice if we could look at the sample image before choosing a diagnosis. 


1017 


We did not look at enough abnormal scans to be able to successfully interpret abnormal findings 


1021 


with a good picture quality it is adequate 


1022 


video adequate, live patient practice was inadequate 


1023 


some difficult cases allowed continuous feedback our our dx skills 


1025 


I did not see enough normals/abnormals to feel confident. Also I had no idea how to determine 
severity (minimal, moderate, severe). 


1026 


Just need to be able to get a better picture & see more examples to be more proficient. 


1027 


Need more time to continously experience multiple examples. 


1028 


Again, no feedback, I didn't know the diagnostic criteria for fluid quantity. Hard to differentiate 
among minimal/moderate/severe. 


1029 


I wasn't sure in my diagnosis= 


1030 


Needed more feedback or examples of abnormal US scans either in the videos or from computer 
simulator. 


1031 


I think this kind of thing takes many years to training to fully be able to identify abnormal vs. normal. 


1032 


Would have been useful to know what we were supposed to find after submitting our images. More 
feedback would be good. 
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ID 



Comment 



1033 Harder to come to diagnosis when difficult to obtain correct window scan. 

1034 It would have been helpful to go over some abnormal images with the trainer 

1035 I felt comfortable noticing abnormalities on adequate images 

1036 There is a lot of anatomical variability. . . practicing on more sick patients would have been helpful 

1037 Although, I wasn't sure to call what appeared to be ascites anterior to liver/kidney abnormal— it is but 
not sure if it was asking for Morrison's pouch as abnormal 

1038 Couldn't see abnormal on real patient. Need more. 

1039 I need to see more abnormal. 

1040 Need to see quite a bit of normals prior to feeling super confident about an abnormal 

1041 For the three views trained, I felt comfortable. However the paper exam had many images I could not 
identify. Either these are of different views/organs, or I was inadequately trained. 

1042 Seeing the standard images to compare with what I saw really helped! 

1043 I'm confident, but not yet overconfident. 

1046 Artifacts were not explained sufficiently. 

1047 The simulations provided good practice of recognizing both normal and abnormal cases. 

1049 The simulator did this well. 
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Which part of the training helped you the most to answer the above type (posttest) 



questions? Please explain. 



ID 


Comment 


1017 


both types of training were helpful. 


1021 


mainly due to being a visual learner 


1023 


more didactic & good window was already given to us 


1025 


But both are important! 


1028 


Through trial and error, got to see what are minimal/moderate/severe cases. 


1029 


I felt the picture and the simulator looked similar. 


1031 


You had time to screw around and appreciate the anatoy in the computer simulations, whereas with 
the live patients we were being timed and I was also trying to hurry to minimize patient discomfort. 


1032 


Both parts were critical to understanding ultrasound procedure. 


1033 


There were photos of normal— abnormal conditions. I learned most when I was able to see all of 
those at one time and able to compare and see where differences lie. 


1039 


I remember better if I do it with my hand. 


1041 


I would agree the computer helps more because you can see more images/comparisons. However, 
hands on permits better spatial understanding, which judged essential for interpreting an image. 


1042 


The video helped me understand basics, but the simulator helped me begin to identify the visual 
patterns of the structures. 


1043 


I needed the computer and the practice making judgment calls from the session. 


1045 


good pictures, examples 


1046 


The same image was on computer test. 


1047 


The computer video showed me what to look for and the simulation helped me gain confidence in 
my ability to make accurate diagnoses. 


1049 


The simulator allowed some manipulation of movements. 
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What were the most useful features of the simulator? [Experimental group only] 



ID 


Comment 


1023 


feedback: instructor and self 


1024 


real-time visualization + positioning 


1025 


It was excellent, actually. The repetition, accurate probe, and continually comparing your image to 
standards was very helpful. 


1026 


Just getting used to manipulating the probe in relation to the body. 


1027 


multiple rounds of practice— getting feedback as to correct/incorrect and quality of scan; helpful seeing 
the scan with rays to know where/what you are looking at in the body. The skeleton overlay was great 
for that. 


1028 


Ability to practice manipulating the ultrasound probe in the right positions (getting comfortable with 
US basically); acquiring window practice 


1029 


I could spend a lot of time on each one. 


1030 


good opportunity to learn how to manipulate the probe. 


1031 


The fact that there was somebody to check my work. 


1032 


Being able to see an image very clearly with movement of the probe. 


1033 


1. seeing what adequate scan window was. 2. practice manipulation of probe— especially in small 
movements 


1042 


The normal/abnormal images for comparison. 


1043 


quick cases, easy to pick up 


1044 


the multiple cases of the practice session 


1045 


layers 


1046 


It's sensitivity to motion. The images were similar to live patients. 


1047 


the ability to manipulate the probe in real time 


1048 


the layers, the comparison images 


1049 


Identifying abnormalities 
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How could the simulator be improved? [Experimental group only] 



ID 


Comment 


1023 


let us more inferiorly/superiorly/laterally 


1024 


if the sonographer could give comments. Also, having less concentration on severe v. moderate v. 
minimal, more important to identify (+ measure with ruler?) fluid collection than give exact severity 


1025 


can't think of anything. It's great! 


1026 


More time with the simulator. 


1027 


At first, having a fixed scan position is great for getting the hang of it. But perhaps with time, the 
simulator could make you place the probe yourself. 


1028 


At the moment the simulator can only be rotated to different views but cannot be manipulated (moved) 
superior/anterior when doing RUQ/LUQ exam and medial/lateral during suprapubic exam 


1030 


allow translational movement in real space 


1031 


Make the user responsible for positioning of the transducer on the patient's body. 


1032 


Allow for lateral movement of the probe, rather than just "fanning." 


1033 


It's good as is. It was fun & very informative. If they can give answer at the end of each case and 
outlined organs and abnormality— that would be very educational. 


1042 


In addition to the "normal" and "abnormal" standard images, also show a "best window" image for that 
same case! This would help me know what I could have done better (& label it!) 


1043 


Don't limit the field of view based on what's right. 


1044 


perhaps using a fake human body so we can orient ourselves better 


1045 


change gain, more accurate with respect to layers, shadowing. 


1046 


Provided as an adjunct to class. 


1047 


provide analyses of what the student should see after each case 


1048 


ability to see orientation of probe for RUQ, larger probe size (similar to real) 


1049 


more movements & different pressures. No fixed axis. 
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What would have helped you improve your performance conducting the FAST scan on 
today’s patients? [Experimental group only] 

ID Comment 

1023 the expert allowed to teach us as we go 

1024 more time to practice, also trying at least once on a real patient first 

1025 I think the best way to instruct med students would be to do the simulator and then receive some live 
teaching. The simulator really helps but I was unsure where to place the probe, how to solve problems, 
and how hard to press. I had a lot of questions and had no one to answer them. 

1026 Again a little more time with hands-on teaching while using the simulator 

1027 More time. Simulator was great, just needed more practice. More "real" patient practice, too. 

1028 Having an instructor there telling me if I have acquired appropriate image, since I have so little 
experience with US. 

1029 more practice on the simulator 

1030 knowing history of patient 

1031 Knowing where to place the transducer and not having somebody standing there timing me. 

1032 Live feedback from a clinician. 

1033 Practice session + teaching session on live patients after simulation. The simulator got us familiar with 
using an ultrasound. What was needed (at least for me) was someone there to instruct & correct me 
when conducting scans on real patient. 

1042 Practicing on a patient some (especially with feedback from an instructor!) 

1043 more patient time 

1045 actual patient 

1046 Having an actual instructor rather than an evaluator 

1047 more practice using the probe without a limited range of motion. 

1048 feedback during exam of real patient 

1049 Practice on patients. Also, didn't realize till part way through the exam that the probe has things that 
look like indicators on both sides, so I had trouble with it. 
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APPENDIX E: 



PICTURE OF A CLASSROOM PRACTICE SESSION 
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APPENDIX F 



PICTURE OF THE LAB SETUP 
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APPENDIX G 



CLASSROOM PRACTICE OBSERVATION FORM 



(Classroom Practice Observation Sheet 

Date / Time: / Instructor ID: Researcher: 



ID 


Quadrant 


Time to Scan 


Notes 




□ RUQ 


□ LUQ 


□ Suprapubic 








□ RUQ 


□ LUQ 


□ Suprapubic 








□ RUQ 


□ LUQ 


□ Suprapubic 








□ RUQ 


□ LUQ 


□ Suprapubic 








□ RUQ 


□ LUQ 


□ Suprapubic 








□ RUQ 


a LUQ 


□ Suprapubic 








□ RUQ 


□ LUQ 


□ Suprapubic 








□ RUQ 


□ LUQ 


□ Suprapubic 








□ RUQ 


□ LUQ 


□ Suprapubic 







Page: 1 
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APPENDIX H: 



PERFORMANCE TEST OBSERVATION FORM 



Date / Time : / Participant ID: 

Researcher: 



Ultrasound Machine: □ MicroTurbo □ SonoSite 


Patient Table: □ 1 U2 


Quadrant 


Diagnosis 


Time to scan 


Notes 


RUQ 


□ Normal / negative 

□ Abnormal / positive 

□ Cannot acquire window 






LUQ 


□ Normal / negative 

□ Abnormal / positive 

□ Cannot acquire window 






Suprapubic 


□ Normal / negative 

□ Abnormal / positive 

□ Cannot acquire window 
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APPENDIX I: 



PICTURE OF THE SIMULATION SETUP 
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APPENDIX J: 

SIMULATION USER INTERFACE SCREENSHOTS 




Figure 1. Screenshot of ultrasound simulator (suprapubic quadrant [bladder]). 
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Figure 2. Screenshot of ultrasound simulator (LUQ/spleen) with layers shown. 




Freeze 



Layers 



a Itrasound Trainer - Case 1, Morrison's Pouch. ML-N4S5A 



Figure 3. Screenshot of ultrasound simulator (RUQ/Morrison’s pouch) with layers shown. 
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Figure 4. Screenshot of scans (normal, various positive) that participants could use to compare his or her 
scan. Displayed after participant submitted his or her diagnosis. 
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Figure 5. Screenshot of prompt for the participant’s 
diagnosis and confidence ratings. 
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APPENDIX K: 



SIMULATION CONDITION DIRECTIONS 



Task 

Use the simulator to practice (a) ultrasound scanning 
and (b) diagnosing the presence of free fluid in the 
abdomen. 

You will have 1 hour to practice scanning 10 virtual 
patients. 

For each patient, you will practice scanning and 
diagnosing three cases: 

Case 1 : Morrison’s Pouch 

Case 2: Suprapublic 

Case 3: LUQ (left upper quadrant) 

Feedback 

An expert sonographer will evaluate your diagnosis, 
the quality of your scan window, and determine 
whether you can move to the next case. 

The sonographer will not explain the evaluation or 
provide any other feedback - this is intentional and part 
of the research study. 

Remember to carefully compare your scan with the 
4 computer-generated scans showing 

(a) negative/normal 

(b) positive/moderate 

(c) positive/severe 

(d) inadequate to interpret 
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APPENDIX L: 



ULTRASOUND SIMULATION REPORT 



1046_20100918_123758_4C951516_0023.pdf 



1046 
Trial 1 



Procedure: Case 6, Spleen, SL-N3S2A 




STUDENT EVALUATION 
Diagnosis: Negative (normal) 

Window confidence level: 

Very confident 

Diagnosis confidence level: 
Very confident 



Session start time: 
09/18/10 12:37:58 


SONOGRAPHER FEEDBACK 




Trial start time: 


Diagnosis: 


Window quality: 


09/18/10 13:37:40 


Q Correct 


^Excellent 


Trial end time: 


Q Incorrect 


□ Fair 


09/18/10 13:38:16 


10 Other 


□ Poor 


Time to complete: 
00:00:36 




n Other 
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APPENDIX M: 



SONOGRAPHER FEEDBACK FORM 
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1025 

Patient 1: Normal 




Sonographer Feedback: 



Diagnosis: 

□ Correct 

□ Incorrect 

□ Other 



Diagnosis: 

□ Correct 

□ Incorrect 

□ Other 



Diagnosis: 

□ Correct 

□ Incorrect 

□ Other 



Window Quality: 

□ Excellent 

□ Fair 

□ Poor 

□ Other 



Window Quality: 

□ Excellent 

□ Fair 

□ Poor 

□ Other 



Window Quality: 

□ Excellent 

□ Fair 

□ Poor 

□ Other 



